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EXECUTIVE SUMMARY 



Job Corps is a major part of federal efforts to provide education and job training to 
disadvantaged youths. It provides comprehensive services: basic education, vocational skills 
training, health care and education, counseling, and residential support. More than 60,000 new 
students between the ages of 16 and 24 enroll in Job Corps each year, at an annual cost to the federal 
government of more than $1 billion. Currently, the program provides training at 1 19 Job Corps 
centers nationwide. The National Job Corps Study is being conducted under contract with the U.S. 
Department of Labor to provide Congress and program managers with the information they need to 
assess how well Job Corps attains its goal of helping students become employable, productive 
citizens. 

This report is one of a series presenting findings from the study. It presents estimates of the 
impacts of Job Corps on participants’ literacy and numeracy skills needed to function in the 
workplace. It builds on the analysis and findings presented in our report on short-term impacts 
(Schochet et al. 2000). That report relied on interview data collected at baseline, and at 12 
and 30 months after random assignment. The current report is based on an in-person literacy 
skills assessment administered to some sample members in conjunction with the 30-month follow-up 
interview. These test score data allow us to measure the extent to which Job Corps improves the 
functional literacy and numeracy skills of Job Corps participants. 



STUDY DESIGN, DATA, AND METHODS 

The cornerstone of the National Job Corps Study was the random assignment of all youths found 
eligible for Job Corps to either a program group or a control group. Program group members were 
permitted to enroll in Job Corps and control group members could not (although they could enroll in 
other education or training programs). The research sample for the study consists of approximately 
9,400 program group members and 6,000 control group members randomly selected from among 
the nearly 81,000 applicants nationwide who applied to Job Corps for the first time between 
November 17, 1994 and December 16, 1995 and who were found eligible by February 1996. 

The study to measure Job Corps impacts on participants’ literacy and numeracy skills was based 
on a randomly selected subsample of program and control group members. Key features of the study 
design are as follows: 



• A single round of skills measurement was conducted in conjunction with the 30-month 
follow-up interview. Because of limited study resources, only a single round of testing 
could be conducted. We selected the 30-month measurement point under the assumption 
that the most important study goal is knowing whether and to what extent Job Corps 
produces differences after most program participants leave the program and spend some 
time in the workforce. 
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• ^ total of 3,750 sample members (1,875 program group and 1,875 control group 
members) was randomly selected for the literacy study. The sample was selected from 
all program and control group members who were eligible for 30-month interviews and 
who were randomly assigned during the last 7 months of the 16-month sample intake 
period. The analysis sample contains 1,117 program and 1,156 control group members 
who completed the literacy test. The overall weighted response rate was 60.2 percent 
and was similar for program and control group members. On average, respondents 
attempted 85 percent of the tasks they were asked to perform. 

• The approach to literacy assessment developed by the Educational Testing Service 
(ETS) was used for the study. Specifically, this study used a version of the assessment 
instrument that ETS designed for the National Adult Literacy Survey (NALS). We also 
considered a range of other instruments, including the Test of Adult Basic Education 
(TABE), which the Job Corps academic education program uses as a diagnostic tool for 
program participants. We selected the instrument designed by ETS, however, because it 
focuses on functional literacy and numeracy skills rather than academic skills only. 
Assessing functional skills is meaningful because Job Corps’ mission is to prepare its 
students for a job or for further education that will lead to a job. The ETS approach has 
been used also in several national studies with populations similar to the population of Job 
Corps students. 



Program impacts were estimated by comparing the average test scores and the test score 
distributions of program and control group members. We estimated program impacts for the full 
sample and for key subgroups defined by the following baseline characteristics: gender, presence 
of children, age, educational attainment, and residential designation status. 



HOW WE ASSESSED LITERACY SKILLS 

The approach to literacy assessment developed by ETS measures the ability to perform a wide 
range of information-processing tasks that adults encounter in everyday life. The approach posits 
three dimensions of literacy: 



1 . Prose literacy, the knowledge and skills necessary to understand and use information 
from texts 

2. Document literacy, the knowledge and skills necessary to locate and use information 
in tables, charts, graphs, and maps 

3. Quantitative literacy, the knowledge and skills necessary to perform different arithmetic 
operations using information embedded in prose and document materials 
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Proficiency in each of these domains is measured on a scale from 0 to 500. To estimate 
proficiency, ETS developed a large number of tasks of widely varying difficulty. Test takers are 
asked to attempt randomly chosen subsets of the tasks. Test results are then used to estimate 
proficiency levels for a population group of interest (the program and control groups in our case). 
A member of a group with a proficiency score of 290 on the prose scale, for example, has an 80 
percent probability of correctly completing a prose task at the 290 difficulty level. This same person 
has a lower probability of completing more difficult tasks and a higher probability of completing less 
difficult ones. 

To facilitate descriptions of the literacy scores of groups and cross-group comparisons, the ETS 
approach to assessment distinguishes five broad literacy levels. Scores below 225 represent the 
lowest level of proficiency. Tasks in this range include locating a piece of information in a simple 
form or document. Scores between 225 and 275 represent Level II proficiency. Level II tasks 
include locating a piece of information in a more complex document with a distractor or performing 
a simple calculation with numbers easily found in a document. Scores between 275 and 325 
represent Level III, scores between 325 and 375 represent Level IV, and scores between 375 and 500 
represent Level V. A score in Level V indicates advanced skills in performing a variety of tasks that 
involve the use of complex documents. 



LITERACY SKILLS OF ELIGIBLE JOB CORPS APPLICANTS COMPARED TO THOSE 
OF OTHER YOUNG ADULTS 

The typical youth served by Job Corps has lower functional literacy scores than the typical 
young adult in the U.S., especially in the quantitative literacy domain. The average proficiency 
scores of control group members were 248 for prose, 256 for document, and 23 1 for quantitative 
literacy. In comparison, young adults nationally averaged 280 points on the prose and document 
scales and 277 on the quantitative scale, as reported in the NALS. While 14 percent of young adults 
nationally performed at the lowest skill level in prose and in document literacy, 28 percent of Job 
Corps control group members scored at that level on prose and 20 percent on document literacy. For 
quantitative literacy the gap was considerably wider. About 16 percent of young adults nationally 
scored in Level I on the quantitative scale, compared to 44 percent of Job Corps control group 
members. 

Part of the reason for these deficits is that Job Corps applicants have considerably lower levels 
of educational attainment than the general population. Looking within education level, the 
Job Corps sample more nearly resembled the NALS sample in literacy proficiency. Remaining 
differences could be explained by a variety of factors. For example, high school dropouts who apply 
to Job Corps are more skilled in nonquantitative areas than dropouts in the general population, 
possibly reflecting a greater level of ability among dropouts who apply to Job Corps than dropouts 
generally. 
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IMPACTS ON LITERACY SKILLS 



The impacts of Job Corps on participants’ functional literacy skills were positive in all three 
domains (see Table 1). Job Corps raised participants’ average test scores by about 4 points on the 
prose scale, 2 points on the document scale, and 5 points on the quantitative scale. The impacts on 
prose and quantitative literacy are statistically significant (different from zero) at the 10 percent 
level. 

Program impacts on tests scores are often expressed as effect sizes or changes in percentile 
ranking among the full population. The effect size is the fraction of a standard deviation, which in 
the case of the literacy assessment used in this study is about 40 points. The effect sizes of the 
estimated impacts on Job Corps participants are about 0.09 for prose literacy, 0.04 for document 
literacy, and 0.10 for quantitative literacy. Taking someone who scores at the 50th percentile of a 
distribution, these effect sizes correspond to increases to the 53rd, 52nd, and 54th percentile of that 
distribution. 

In terms of discrete proficiency levels, Job Corps moved some participants out of Level I. 
About 3 percent of participants are estimated to have moved out of the lowest proficiency level on 
the prose scale (2 percent on the document scale and 5 percent on the quantitative scale) and a 
comparable fraction into Levels II and III. Again, this effect is statistically significant for prose and 
quantitative literacy, but not for document literacy. Very few members of the program or control 
groups scored in the top two proficiency levels in any of the three literacy scales. 

Positive impacts were found broadly across most key subgroups of students. Nearly all of the 
impacts estimated at the subgroup level were between 3 and 6 points per Job Corps program 
participant, although most are not statistically significant because of small sample sizes. Estimated 
impacts, however, were somewhat larger for older applicants who did not have a GED or high school 
diploma at random assignment-a group with particularly low skills. Impacts for this group were 
6 points on both the prose and document literacy scales, and 1 1 points on the quantitative scale. 



INTERPRETATION OF THE IMPACT FINDINGS 

In order to interpret the literacy impact findings, we examined the extent to which the estimated 
impacts on literacy skills are consistent with our impact findings on other key outcomes that 
are associated with basic skills. We analyzed the statistical relationships among literacy scores, 
educational experiences, and labor market experiences for our basic skills sample by combining 
information on their test scores with information from their baseline and follow-up interviews. We 
then tried to reconcile the findings. 

The analysis relied on a simplified path model of the relationship between family background, 
schooling, literacy skills, and labor market outcomes. Literacy is considered to be an intermediate 
(mediating) outcome that is affected by schooling and work experience, and that affects later 
outcomes such as earnings. 
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